Scalable Logging through Emerging Non-Volatile Memory

Emerging byte-addressable, non-volatile memory (NVM) is fundamentally changing the design principle of transaction logging. It potentially invalidates the need for flush-beforecommit as log records are persistent immediately upon write. Distributed logging—a once prohibitive technique for single node systems in the DRAM era—becomes a promising solution to easing the logging bottleneck because of the nonvolatility and high performance of NVM. In this paper, we advocate NVM and distributed logging on multicore and multi-socket hardware. We identify the challenges brought by distributed logging and discuss solutions. To protect committed work in NVM-based systems, we propose passive group commit, a lightweight, practical approach that leverages existing hardware and group commit. We expect that durable processor cache is the ultimate solution to protecting committed work and building reliable, scalable NVM-based systems in general. We evaluate distributed logging with logging-intensive workloads and show that distributed logging can achieve as much as ∼3x speedup over centralized logging in a modern DBMS and that passive group commit only induces minuscule overhead.